About

Avinash B Soppimath is a certified Data Engineer with over 3 years of experience in building ETL and ELT pipelines using Apache Spark on Azure Databricks and implementing delta lakes in the retail space. He is currently working as a Data Engineer at Gap Inc. and is proficient in building frameworks involving data integrity checks, data visualizations, pipeline and data auditing, building reconciliation reports, data movement, transformations, reporting, data synchronization, etc. Avinash has worked on multiple orchestration tools such as Apache Airflow and Azure Data Factory. He has also worked as a Specialist Programmer at Infosys, where he built multiple data pipelines, both ELT and ETL, on the Azure platform involving complex datasets with evolving payload structures leveraging ADF and Azure Databricks. He has worked on multiple pipeline migrations involving Re-platform, Re-factor, and Re-architect approaches. Avinash has also developed generic metadata-driven frameworks that helped teams in data ingestion, pipeline auditing, building dashboards for monitoring pipelines and data, reconciliation with alerting, cross-platform data synchronization, and PI I data encryption. He has also developed on-the-fly query builders that helped teams in faster development of pipelines. He has worked on pipeline orchestrations leveraging Apache Airflow, ADF, and Azure Databricks Jobs. Avinash holds a Bachelor of Engineering degree in Computer Science from R. V. College of Engineering, Bangalore. He also has a background in Computer Science from Vikas Pre-University College and Don Bosco School. His tech stack includes Data Engineering, Azure, Databricks, Apache Airflow, Python, Spark, ETL, SQL, NoSQL, MongoDB, Senior Software Engineer, and Mobile. With relevant experience of 3.45 years, Avinash is a skilled Data Engineer with a passion for building efficient and scalable data pipelines.